Sonification of biometric data state-songs generation, biological stimulation modelling and artificial intelligence

ABSTRACT

Embodiments are generally directed to methods, systems and devices for generating and playing back a personalized state song representing a target state of a user. In one scenario, a computer system accesses biometric data corresponding to various bodily systems of a user. The computer system encodes the biometric data into data structures that are configured for modeling using an algorithm that is specific to the user. The computer system models the encoded biometric data to identify specified patterns in the encoded biometric data, and these identified patterns are used to generate a personalized algorithm that corresponds with the identified patterns unique to the user. The computer system then synthesizes a state song using the personalized algorithm, which represents a specific biometric state for one of the bodily systems, and plays back the state song to the user to induce the user to the specific biometric state.

CROSS-REFERENCE TO RELATED APPLICATIONS

This CIP application claims the benefit of, and priority to, U.S. patentapplication Ser. No. 15/405,744, entitled “SONIFICATION OF BIOMETRICDATA STATE-SONGS GENERATION, BIOLOGICAL STIMULATION MODELLING ANDARTIFICIAL INTELLIGENCE,” filed on Jan. 13, 2017, which applicationclaims priority to and the benefit of U.S. Provisional PatentApplication Ser. No. 62/388,076, entitled “SONIFICATION OF BIOMETRICDATA STATE-SONGS GENERATION, BIOLOGICAL STIMULATION MODELLING ANDARTIFICIAL INTELLIGENCE,”and filed on Jan. 15, 2016, both of whichapplications are incorporated by reference herein in their entirety.

BACKGROUND

An individual's “states” (i.e., happy, depressed, fearful) embody “statedependent behaviors” (smiling, inactivity, lashing out). By examples:persons suffering from PTSD are suspended in a state of trauma, andwithin that traumatized state, a set of negatively reactive behaviorsare expressed. Outside of the traumatized state, the set of PTSDbehaviors are less accessible. More positively, behaviors related to acalm state generally exclude the actions associated with trauma, stressand anxiety. Ideally, one would be able control his/her emotions so asto achieve positive states and sustain constructive behaviors.Unfortunately, the control of one's states can be elusive.

There is thus a need for systems and methods for more effectivelycontrolling state dependent behaviors of individuals.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features, aspects, and advantages of the presentdisclosure will become better understood regarding the followingdescription, appended claims, and accompanying drawings.

FIGS. 1-3 illustrate different external and internal states for context.

FIG. 4 illustrates a method for controlling state dependent behaviors ofa user according to an embodiment.

FIG. 5 illustrates a system for controlling state dependent behaviors ofa user according to an embodiment

FIG. 6 illustrates a somatic module according to an embodiment.

FIG. 7 illustrates an endocrine module according to another embodiment.

FIG. 8 illustrates an EEG module according to another embodiment.

FIG. 9 illustrates a system of module integration according to anembodiment.

FIG. 10 illustrates a modelling system according to an embodiment.

FIG. 11 illustrates a modelling system according to another embodiment.

FIG. 12 illustrates a user interface according to an embodiment.

FIG. 13 illustrates a user interface according to another embodiment.

FIG. 14 illustrates a user interface according to another embodiment.

FIG. 15 illustrates a computing architecture in which embodimentsdescribed herein may operate.

FIG. 16 illustrates a flowchart of a method for generating and playingback a personalized state song representing a target state of a user.

FIGS. 17A and 17B illustrate a system for generating and playing back apersonalized state song representing a target state of a user.

BRIEF SUMMARY

Embodiments described herein are generally directed to methods, systemsand devices for generating and playing back a personalized state songrepresenting a target state of a user. For example, in one embodiment, acomputer system accesses biometric data corresponding to various bodilysystems of a user. The computer system encodes the biometric data intodata structures that are configured for modeling using an algorithm thatis specific to the user. The computer system models the encodedbiometric data to identify specified patterns in the encoded biometricdata, and these identified patterns are used to generate a personalizedalgorithm that corresponds with the identified patterns unique to theuser. The computer system then synthesizes a state song using thepersonalized algorithm, which represents a specific biometric state forone of the bodily systems, and plays back the state song to the user toinduce the user to the specific biometric state.

DETAILED DESCRIPTION OF VARIOUS EMBODIMENTS

A better understanding of different embodiments of the disclosure may behad from the following description read with the accompanying drawings.

While the disclosure is susceptible to various modifications andalternative constructions, certain illustrative embodiments are in thedrawings and are described below. It should be understood, however,there is no intention to limit the disclosure to the specificembodiments disclosed, but on the contrary, the intention covers allmodifications, alternative constructions, combinations, and equivalentsfalling within the spirit and scope of the disclosure.

It will be understood that unless a term is expressly defined in thisapplication to possess a described meaning, there is no intent to limitthe meaning of such term, either expressly or indirectly, beyond itsplain or ordinary meaning.

Any element in a claim that does not explicitly state “means for”performing a specified function, or “step for” performing a specificfunction is not to be interpreted as a “means” or “step” clause asspecified in 35 U.S.C. §112(f).

Embodiments of the present disclosure transform biometric data from auser into sound which is fed back into the user to beneficiallyinfluence state dependent behaviors of the user. For instance, methodsof the present disclosure can involve generating biometric based musicto induce a desired or target internal state of clam, learning, and/orsecond wind in the user. Alternatively, methods described herein caninvolve generating biometric based music for use in a movie ortelevision soundtrack to induce one or more desired audience emotions.In other embodiments, methods of the present disclosure can be adaptedto provide an alternative mode and methodology for biologicalstimulation modelling. In yet other embodiments, methods of the presentdisclosure can be adapted to generate biometric based music and utilizethe same as a predictive, diagnostic, and/or artificial intelligencetool as described in more detail below.

FIGS. 1-3 illustrates exemplary relationships between internal andexternal states of a user to provide context to the present disclosure.Referring to FIG. 1, an internal state A comprises the physiologicalstate of depressed, and an external state B comprises a physical postureB, which when depressed, includes slumped shoulders, chin pointing down,low effect. Internal state A of depressed and the physical posture B aregenerally isomorphic or two different presentations of the same state atdifferent levels, internal and external. The physical posture B isgenerally a result of the individual's internal state or physiology. Forinstance, if the internal physiology is depressed the posture physicalpresents as such.

Referring to FIG. 2, the relationship between the internal state A andexternal state or physical posture B can be reciprocal in that thephysical posture B feeds back upon and impacts the internal state A insuch a way that if we change the physical posture B by pulling theshoulders back, pushing the chin upwards and projecting a posture ofconfidence, the internal state A conforms in accordance with thephysical posture B of confidence to change to an internal state ofconfidence. In other words, the internal state A creates the physicalposture B but the physical posture B can influence or force the internalstate A to conform to the physical posture B. Thus, while the externalstate or physical posture B is a postural isomorph derived from withinthe internal or physiological state A, the external state or physicalposture B can control the internal state A from the outside-in.Cybernetics dictates that if the external state B is fed back into thesystem of the internal state A from which it was derived, the internalstate A will at least in part conform to the external state B.

Referring to FIG. 3, the described relationship between the internalstate A and the external state or physical posture B can be extended toinclude other isomorphs such as a sonic isomorph. Biometric based musiccan be derived from a system in an internal state A and then fed backinto the system to influence the system. The biometric based music canbe referred to as a state-song C. This state-song C represents a stateof a system of origin and can cause the internal state A of that systemto at least in part conform to the state-song C. For instance, if thatstate-song C is generated from biometric data obtained from a user withan elevated heart rate that user can listen to the state-song C at adifferent time when the user's heart rate is reduced and experience apalpable increase in its heart rate.

Systems and methods of the present disclosure can advantageously thusgenerate biometric based music from biometric or physiological data tocreate a sonic isomorph of a state that induces that same or similarstate. This can be identified as a phenomenological isomorph meaningthat the same result can be achieved with other state isomporphs (e.g.,music). Similar to the previous example of the physiology/posture case,the state-song C derived from a specific internal state can be fed backinto an user or system of origin to induce a targeted state; such ascalm, aware, excitement, second wind, focus, etc.

Systems and methods of the present disclosure can be adapted to addressPTSD, depression, anxiety, insomnia, and/or any other internal state soas to modify the physiology and behaviors that are encumbered by thosestates. Other embodiments, can be adapted to produce individualizedsonic simulation models that merge somatic, EEG and endocrinologicaldata. Alternatively, such personalized simulation models can be used byartificial intelligent agents for health diagnostics and interventions.

FIG. 4 illustrates exemplary steps in a method 50 for controlling statedependent behaviors of a user. In an act 60, biometric data is obtainedfrom a user. The biometric data can include somatic data, endocrinedata, cortical data, or any other suitable biometric data.

In an act 70, at least some of the biometric data is converted ortransformed into lines of sound. This may be referred to as “sonifying”the biometric data. Conversion or transformation of the biometric datamay include frequency/amplitude conversion and/or algorithmicprocessing. The conversion and/or transformation of the biometric datacan be performed by one or more modules as discussed below. Forinstance, the biometric data can be converted or transformed into linesof sound by a processing module of a computing device.

In an act 80, at least some of the lines of sound are compiled intocompositions or state-songs arranged to represent a targeted state ofthe user. The conversion and/or transformed of the biometric data can beperformed by one or more modules as discussed below. The targeted statescan include different states such as calm, learning, exercise, rest,etc.

In an act 90, the compilation or state-song is fed or provided back tothe user to induce the user to the targeted state. The compilation orstate-song can be fed or provided to the user via an output componentsuch as a speaker, headphones, or any other suitable output device.

It will be appreciated that method 100 can adapted or implemented tohelp reduce suffering from post-traumatic stress disorder (PTSD),Anxiety, Obesity, ADHD, Schizophrenia, and/or other psychologicalconditions.

According a variation, the state-songs of the present disclosure canfacilitate predictive analytics and personalized medicine. For instance,a user's state song can be singularly unique to that individual. From aclinical perspective this helps move health sciences past groupstatistics toward modelling, monitoring and intervention of a highlyindividualized and predictive nature. Furthermore, the parallel imbeddednature of biometrics is rich soil for deep learning. Aimed towardsbio-simulations with individual accuracy, methods of the presentdisclosure can define the way intelligent agents learn, diagnose,predict, prevent and intervene. In addition, integration of state songsand holography may provide a platform on which to evolve and utilize atruly intelligent AI.

It will be appreciated that any of the methods described herein may beimplemented in an application. The application may be software embodiedon a computer readable medium which when executed by a module of acomputer system performs a sequence of steps. The application may be amobile application or application software configured to run onsmartphones, tablets computers, and/or other mobile devices. Theapplication may be a web-based programming language and/or web-basedcomputing platform. The application may be a computer programminglanguage that is concurrent, class-based, object-oriented, and design tohave minimal implementation dependencies (e.g., Java programminglanguage).

FIG. 5 schematically depicts a system 100 for controlling statedependent behavior of a user. The system 100 may include a computerdevice that can display information to a user and receive user input,respectively. The computer device can include a mobile device. A mobiledevice is defined as a processing device routinely carried by a user. Ittypically has a display screen with touch input and/or a keyboard, andits own power source. The system 100 may include biometric data capturedevices (e.g., sensors), storage devices, and/or transmission devices.

The system 100 can be in communication with an application and/or acloud computing platform. The application and/or cloud platform can beconfigured to perform any of the acts described herein. For instance,the cloud platform can be arranged to convert data (e.g., biometricdata) to lines of sound, sounds and/or songs. In other embodiments, thecloud platform can be arranged to store and/or transmit data, sounds,and/or songs.

In other embodiments, the cloud platform and/or application can bearranged to manage intelligent agents and/or artificial intelligence.For instance, the cloud platform and/or application can includepredictive simulation and/or health diagnostic agents. In an embodiment,the application and/or cloud platform can be arranged to performsonificiation and/or include holographic intelligence systems.

The system 100 may include different modules 102-118 arranged to performdifferent functions. For instance, the system 100 may include one ormore modules for biometrics data sonification, biometrics and stateanalysis, sound generations, state-song composition, state-song storageand access, user services and tools, biological systems modelling,and/or intelligent agents. According to a variation, the system 100 caninclude an ingest module 102 arranged to receive and/or store dataincluding biometric data. The biometric data may include any suitabletype of data, including, but not limited to, somatic data, EEG data,endocrine data, and/or user defined data. The system 100 can alsoinclude an encode module 104 arranged to encode or sonify data from theingest module. The system 100 can include an MM/SM module 114 includingplayback devices, analytics and info, editing tools, data and infosecurity, simulation models, and/or intelligent agents. In otherembodiments, the system 100 may include input/output modules includingdashboards and/or graphical user interfaces.

FIG. 6 illustrates a somatic module 120 according to an embodiment. FIG.7 illustrates a somatic module 130 according to an embodiment. FIG. 8illustrates an EEG module 140 according to an embodiment. FIG. 9illustrates a method 150 of module integration. FIG. 10 illustrates amethod 160 of biometrics and sonimodi-modelling. The method 160 includesstacking modules with data integration and comparisons towardsindividualized medical models. FIG. 11 illustrates a method 170 of statesong, simulation modeling and AI. The method 170 includes stackingmodules with data integration and medical modelling toward biologicalintelligence.

It will be appreciated that the system 100 can include one or more userinterface through which the user is able to input or receiveinformation. For instance, FIG. 12 illustrates a user interface 180implemented on a mobile device 182. As seen, the user interface module180 can be simplified to improve ease of use. FIG. 13 illustrates a userinterface 190 implemented on a mobile device 192. The user interface 192can include more options and controls for a more involved user. FIG. 14illustrates two different user interfaces 200 a and 200 b implemented ona desktop computer 202. The user interfaces 200 a and 200 b can have amore a complex architecture adapted for analytics and modeling.

FIG. 15 illustrates a computer architecture 1500 in which at least oneembodiment described herein may be employed. The computer architecture1500 includes a computer system 1501. The computer system 1501 includesat least one processor 1502 and at least some system memory 1503. Thecomputer system 1501 may be any type of local or distributed computersystem, including a cloud computer system. The computer system 1501includes modules or layers for performing a variety of differentfunctions. For instance, communications module 1504 may be configured tocommunicate with other computer systems. The communications module 1504may include any wired or wireless communication means that can receiveand/or transmit data to or from other computer systems. Thecommunications module 1504 may be configured to interact with databases,mobile computing devices (such as mobile phones or tablets), embedded orother types of computer systems.

Computer system 1501 further includes an encoding layer 1505. Theencoding layer may be configured to receive biometric data 1516 from auser (e.g. 1515) or from a computing device connected to the user. Forexample, the biometric data 1516 may be any type of data relating to auser's body including somatic data such as heart data (heart rate),lungs data (respiration rate), skin data (stress levels), vascularsystem (blood pressure), kinetic information (body position), endocrinesystem data, electroencephalogram (EEG) data, personal information(calendar, activities, relations, concerns, etc.) and other types ofdata. As shown in FIG. 17, each of these different types of biometricdata may be captured and provided to an encoding layer or trans-codingengine.

The encoding layer 1505 of computer system 1501 analyzes the receivedbiometric data 1516 and encodes the data into data structures that canbe modeled in the modeling layer 1507 using an algorithm that isspecific to the user. Indeed, when the modeling layer 1507 models theencoded biometric data 1506, it identifies patterns 1508 that are uniqueto the user 15151. These patterns may be found in rhythmic recurrencesof frequencies, tones, repeated highs and lows in certain measurements,or other patterns. These patterns match a user uniquely, and thus theuse of these patterns to create a state song will result in a state songthat is specific and unique to the user. The patterns 1508 are used togenerate an algorithm that is fed to the synthesis layer 1510. Thealgorithm may represent one type of biometric data, or may representmany different types of biometric data.

The synthesis layer 1510 uses this personalized algorithm 1509, alongwith the encoded biometric data 1506 and generates a state song 1511specific to a given biometric state 1512. For example, if the user 1515is currently in a relaxed state, as evidenced by one or more forms ofbiometric data (e.g. low heart rate, low blood pressure, low stress,etc.), then the state song generated will correspond to the relaxedstate. Many other states may be captured based on the biometric data.The synthesis layer may use the personalized algorithm along with one ormore portions of music data or music knowledge to generate a song orsoundscape that is pleasing to the ear. The playback module 1513 playsback the generated state song 1511 to the user 1515 to bring that userto the specified biometric state. For instance, if the user is agitated,the state song corresponding to a relaxed state may be played back tothe user to bring the user to the relaxed state. The user's biometricdata 1516 may be stored in a local or remote database 1520, along withany state songs 1511 that are generated. These concepts will beexplained further below with regard to method 1600 of FIG. 16, alongwith the computing architectures shown in FIGS. 17A and 17B.

In view of the systems and architectures described above, methodologiesthat may be implemented in accordance with the disclosed subject matterwill be better appreciated with reference to the flow charts of FIG. 16.For purposes of simplicity of explanation, the methodologies are shownand described as a series of blocks. However, it should be understoodand appreciated that the claimed subject matter is not limited by theorder of the blocks, as some blocks may occur in different orders and/orconcurrently with other blocks from what is depicted and describedherein. Moreover, not all illustrated blocks may be required toimplement the methodologies described hereinafter

FIG. 16 illustrates a flowchart of a method 1600 for generating andplaying back a personalized state song representing a target state of auser. The method 1600 will now be described with frequent reference tothe components and data of environment 1700 of FIGS. 17A and 17B.

Method 1600 includes accessing one or more portions of biometric datacorresponding to one or more bodily systems of a user (1610). Forexample, the communications module 1504 may receive biometric data 1516from various electronic devices including bodily system monitors, smartwatches, EEG machines or other devices that produce biometric data. Thedata may be received via wired or wireless data connections.Alternatively, the encoding layer 1505 may directly access the biometricdata 1516, for example, in cases where the data is stored on thedatabase 1520.

The encoding layer encodes the biometric data 1516 into data structuresthat are configured for modeling using an algorithm 1509 that isspecific to the user (1620). The modeling layer 1507 models the encodedbiometric data to identify specified patterns 1508 in the encodedbiometric data. The identified patterns are used to generate apersonalized algorithm 1509 that corresponds with the identifiedpatterns unique to the user (1630). Method 1600 further includessynthesizing a state song 1511 using the personalized algorithm, wherethe state song represents a specific biometric state for at least onebodily system (1640), and playing back the state song to the user toinduce the user to the specific biometric state (1650).

As shown in FIG. 17A, Sections 01-05 depict example data types anddevices that the system (1700 generally) can capture for encoding. Itshould be noted that the modules of the system shown are not limited tothose depicted herein, and it will be understood that the embodimentsdescribed herein may be used to encode data from any biometric sourceinto a state song. Moreover, the biometric data may be leveraged forparallel or other modeling and intelligence implementations (please seediscussion of FIG. 5 above).

System 1700 thus converts biometric data from any device or applicationinto music. This includes, but is not limited to heart rate, bloodpressure, galvanized skin response, respiration rate, body temperature,body velocity, position, etc. The application layer, as depicted inSection 06, is configured to interface with any application via softwaredevelopment kits (SDKs), APIs, etc.

Section 02 depicts data the endocrine system, which may be cyclical.System 1700 can convert endocrine data from any device or applicationinto music. This includes existing, methods that capture and recordhormone and other biological compound levels that can be measured fromblood, lymph or spinal fluid.

Section 03 depicts EEGs that show brain activity. The system 1700converts electroencephalograph data from any device or application intomusic. This includes methods and devices that capture and record brainAlpha, Beta, Delta, Gamma and Theta waves, as well as methods anddevices that capture and record neuronal interference or spectral datafrom the cortex, cerebellum or mid-brain structures via external skullcaps, electrodes, neural fabrics, or other brain-computer interfaces.

Section 04 depicts person, place and time data including calendar andlocations data from any device or application into music. This includemethods and devices that capture and record calendar elements,activities, relationships, reflections, assertions, concerns,affirmations and outcomes, or other information that can be representedquantitatively regarding a person's location, a person's plans, concernsor other activities that could affect the person's biometric state.

Section 05 depicts biometric data capturing applications which gatherdata and render graphs. The biometric data capturing applications can beintegrated with SDKs that allow data to be captured by the system'sapplication layer via API, REST, SOAP, XML, JSON, WSDL or otherprotocols. Many types of biometric data capturing applications exist infields such as science, technology, engineering, medical, and health andwellness. Independent of data capturing method, the system 1700 willintegrate the received biometric data for encoding, and can create avirtual twin model of the person using that data. This will be explainedfurther below.

Sections 06-09 describe system 1700's conversion and GUI layers, as wellas the spectrum of uses across sectors such as health and wellness,fitness, music, medicine and research & development, and the variousplatforms upon which the system can operate to achieve objectives thathave root in state-dependent behavior modification. The embodimentsherein aim to change the user's biometric state using music that isspecifically tailored to the user, and may further be used topotentially alter the user's behaviors. This level of use is primarilyconducted between the user, a biometric device and computer 1501 (i.e.,phone, laptop, tablet, etc.). Access to cloud services and applications,compliant with data protections standards (i.e., HIPPA, PCI, etc.) arediscussed in Sections 10-12 below.

Section 06 includes trans-coding engines & GUIs. The transcoding enginescapture and encode biometric data using algorithmic conversions of thedata that are personalized to the user. The transcoding engines thenascribed sound attributes programmed within the conversion layer. Thislayer can be coded in numerous languages including, but not limited to,ChucK, MaxMXP, PureData and SuperCollider.

Sound attributes include, but are not limited to, instrument selections(piano, strings, shakers, etc.), natural sounds (frogs, drops, wind,etc.), user generated sounds (recorded, programmed), sampling rates, 7.1sound locations, effects (reverb, chorus, compression, phase, etc.) andother sound items. A graphical user interface is provided that grantsthe user access to sound options provided from the transcoding engine,and allows the sound attributes applied to each line of data to becontrolled. The GUI layer can be programmed from a multiple languagesincluding, by example, but not limited to, Java, LISP and HTML5. The GUIlets users take any line of data and multiply it, applying differentattributes to each copy. This allows even a single biometric device todrive multiple lines of sound.

The three devices in Section 06 surrounding the encoding engine and GUIcharacterize examples of the GUI's of varying designs running on aspectrum of computer systems for broad range of purposes, including butnot limited to, 1) health and wellness applications such as sportstraining, track & field, cycling, archery, etc, 2) state and behaviorincluding relaxation, meditation, attention, learning, addiction, 3)music production and distribution, including an application to create‘Personalized’ biometric based music for distribution, and a module forprofessional tools for music, television and film production, as wellsas a platform for medical music including advanced modeling andintelligent agents.

These devices and GUI further provide 4) personalized modeling andvirtual twinning for somatic data (heart rate, respiration rate, bloodpressure, GSR), endocrine data (metabolic, reproductive, stress,immunological data, etc.), neurological, immunological, or data fromother systems. The devices can also provide 5) personalized medicinewith intelligent personal and model monitoring, and intelligentdiagnostic and interventive agents.

Section 07 of FIG. 17A depicts a personal health, wellness and fitnesscategory which exemplifies uses for state song technology in health,wellness and fitness. Some of these uses may include personal fitness,sports improvement, meditation, hypnosis, alleviating anxiety and traumabased disorders, improving learning disorders and disabilities,improving attention disorders and disabilities, generating personalizedmusic, shared and group music for counseling, meditation, concert anddancing music, etc. Biometric devices that can be used to capture datafor generation of state songs includes head and neck devices, chest,back and torso devices, arm devices, wrist devices, hands devices,midsection devices, legs devices, knees devices, feet/shoe devices,headset devices, earbud devices, hands-free devices, medical equipmentthat captures biometric data, etc.

Section 08 depicts a professional category including generation of statesongs that could be used in production for music, television & film.State songs may be used in personal & professional music production onmobile phones, tablets, laptops, desktop computers and othercomputational devices. A state song production module is designed tointegrate with professional audio and video tools. Section 09 issimilar, and is designed to show professional medical and research anddevelopment uses, bio-simulations and AI agents. The system 1700 mayprovide access to artificial intelligence tools that enhance the statesongs and make the songs more appealing to human users. The system alsoprovides personalized simulation modeling and virtual twins. Localizedagents can work offline from the cloud, but can, ultimately, connect todeeper (data protected) functionalities, services, updates, etc throughthe cloud.

Section 10 of FIG. 17B describes a scenario in which biometric and statesong data are stored to the cloud (e.g. database 1520 of FIG. 15). Thesystem 1700 transmits personal and biometric data from any device,application or platform to the state song cloud for storage and stagingfor advance options and functions provided by a service orientedarchitecture (SOA). Data transport and storage is designed to becompliant with relevant standards such as HIPPA and other health-relateddata protection acts. Data is stored, protected and staged for access byauthorized users. The data may be used by advanced encoding and soundtools and applications, advanced modeling tools and applications, andadvanced intelligence tools and applications.

Section 11 depicts a service oriented architecture which is aconfiguration of componentized tools, functions and services accessiblevia the cloud. The services may include: support tools, user anddeveloper tools, access, identity, security and compliance layers, uservirtual studios, sound libraries and music production tool, developmenttools, SDKs, platform and application sandboxes, data ingestion,protection and retrieval, data ingestion and storage, data protection,encryption and standards compliance, and other types of data access.

The SOA of section 11 in FIG. 17B provides encoding tools andapplications including user and developer data encoding tools andapplications, user and developer sound tools and applications, user anddeveloper state song tools and applications, analysis and modeling toolsand applications, data, graphic & sound analysis tools, personal models,model comparisons, concurrent event logs, personal algorithmic dynamicpredictive simulations on a digital virtual twin that has and is basedon the user's same biometric data. Synthesis intelligence tools andapplications including sound analysis and state song enhancements arealso provided by the SOA.

The SOA of Section 11 also provides Data2Mind models, prototypes,applications, platform, holo-infomatic intelligence: diagnostic andinterventive agents, sharing, archiving, monetization, personalizedstate song storage and sharing, state song streaming & sharingplatforms, digital asset management & monetization tools.

Section 12 depicts sonic simulation (isomorphic) modeling. This sonicsimulation depicts multiple representations of an individual'sphysiology data including graphical, numerical, and sonicrepresentations. The representations illustrate how data is monitored,tracked, gathered and graphically depicted. Moreover, therepresentations, as isomorphs, aided by artificial and ‘organic’intelligence, present embodiments where modality models can ‘learn’ fromeach other. Further, these representations can be leveraged, and ‘pushedand pulled’, to provide predictive graphical, numerical and sonicrenderings of the models, and thus the individual's states that itpurports to model.

The biometric and personal data types that can be individually andinteractively modeled are not limited to the list below. Indeed,somatic, endocrinological, neurological, immunological, medications,personal data (activities, reflections, assertions, concerns, other datatypes) can be used to generate highly representative perpetualbiopsychosocial models of an individual. Perpetually gathering biometricdata from bodily subsystems, the system 1700 generates perpetual sonicrepresentations, ‘isomorphs’ (algorithms) of each respective subsystem'sactivities. The ‘integration’ of subsystem algorithms represents alarger algorithmic isomorph' or ‘meta algorithm’ of the individual.

In “Personalized Medicine” parlance, the state song system is providinga new ‘Pairing Technology’, where the results are ‘Virtual Sonic Twins’(VSTs). VSTs can generate state songs from perpetual data gatheringsystems, and use these state songs to monitor, then predict, and ideallyregulate the sonic twin. Then, ultimately, the VST can be applied forutilization with its human twin for monitoring, diagnostics, predictiveanalysis, interventive agents, state control, behavior modification andother uses. In gestalt, the integration of the models is a perpetualrunning algorithm, a ‘virtual twin’ of an individual, upon whichintelligent agents of the system 1700 can analyze, diagnose, predict andact upon the digital twin and subsequently, or in parallel, the human orrobotic twin. The algorithms can control an android, or the functions ofa prosthetic device, or a swarm of robot bees pollinating crops, forexample.

In some embodiments, the system 1700 may be configured to identifyrelationships between different types of biometric data 1516 as part ofmodeling the encoded biometric data 1506. For instance, the system 1700may determine that a relationship exists between heart rate and EEGgamma wave data, or between endocrine cycles and blood pressure, oractivity data, or between any of the variety of different types ofbiometric data. When these relationships are identified, the modelinglayer 1507 may use the relationships when generating the personalizedalgorithm. Thus, the modeling layer 1507 may implement as input theidentified relationships between the different types of biometric dataand the encoded biometric data 1506. Using these relationships and data,the user-specific, personalized algorithm 1509 may be generated.

The encoding performed on the biometric data may be performed at atranscoding engine that includes an encoding layer configured to convertthe biometric data into musical elements by mapping the biometric dataaccording to sampling rate, filters, reverb or compression (see Section06 of FIG. 17A). The musical elements can be notes (i.e. tones of aspecific frequency), sounds of a musical instrument (such as a drum or atrumpet), soundscapes (e.g. rainfall or birds chirping), or any othertype of musical element. In some cases, a default mapping is establishedfor each type of biometric data. For instance, a heartbeat may be mappedto a drum, kinetic movements may be mapped to a lead guitar, EEG thetawaves may be mapped to the sound of waves crashing on a beach, etc.Virtually any type of mapping is possible from any type of biometricdata to any type of sound element.

The mapping may be made according to user input specifying which type ofbiometric data is to be mapped to a specific instrument or soundscape.Additionally or alternatively, an artificial intelligence engine may beimplemented to learn musical patterns that make music acousticallydesirable. The artificial intelligence (AI) engine may analyze manythousands or millions of musical compositions or soundscapes that aredetermined to be acoustically desirable. The AI engine may identifycommon elements among the compositions, and use those elements orpatterns when mapping the different types of biometric data to aninstrument or soundscape. Thus, in this manner, the mapping from encodedbiometric data 1506 to actual music may be performed by an AI engine, ormay be performed according to another program or user input.

Once the state song 1511 is generated, it is played back to the user1515. This playback may be in reaction to the user reaching a specified,undesirable biometric state 1512. For example, a returned soldier mayhave slipped into an undesirable post-traumatic stress disorder (PTSD)state. A state song generated from that user's earlier relaxed state maybe played back to the user to induce him or her to a relaxed state.Alternatively, the state song 1511 may be played back to the user 1515proactively to prevent the user from reaching a specified, undesirablebiometric state. As such, the state song may be used to amp the user upfor an even such as a sport or competition, or may be used to help theuser relax and fall asleep. Professionals may use a state song to inducea period of high brain activity, or to get “in the zone,” or to cleartheir mind of trivial matters. As such, state songs may be used byprofessionals including doctors, lawyers, accountants, engineers,government officials, film actors, athletes, or other types of workers.

At least in some embodiments, this state change in a user's body fromdisheveled to concentrating, or from angry to relaxed, or from lethargicto energetic may be measured and documented. Various devices includingheart rate monitors, blood pressure monitors, endocrine cycle monitors,EEGs or other biometric data gathering devices may be used to measurethe changes in the user that occur as a result of playing back the statesong 1511. Some state songs may be more effective than others forachieving a specified state change. As such, these state songs may beprioritized over others. A user's state songs may also be refined overtime to be more effective at causing the desired change in state.

The biometric data gathering process may also be refined over time.Indeed, as data comes in from various body system monitors, that datamay be filtered for noise. The system 1700 may include a filtering layerthat is configured to filter at least some of the identified noise inthe biometric data. Thus, when the system is identifying patterns in theencoded biometric data 1506, the noise can be identified and filteredout, leaving only the useful biometric data. Once the noise has beenfiltered out, the artificial intelligence layer will have an easier timemapping and providing musical elements as input to the synthesis layer1510 indicating how the state song 1511 is to be generated. Genrefilters may also be applied for the biometric data 1516. For instance, aclassical music filter may be used based on the type of biometric datathat is received, or a rock music filter may be used. This genre filterindicates generally what type of musical elements the AI engine shouldlook for when providing musical elements to the synthesis layer togenerate the state song.

Once created, the state song may be shared with other users. The system1700 includes a sharing layer that provides the generated state song toother users. Many file sharing platforms may be used, including links tothe files stored in the database 1520. In some cases, the state songsfiles are encrypted and are only accessible to authenticated users.Because the state songs are based on biometric data, which is typicallyprotected as private information, the state songs may be stored in alegally compliant manner (e.g. HIPAA-compliant), whether they are storedlocally or on the cloud. The user's biometric data and/or state songsmay be moved to a service oriented architecture (e.g. Section 11 of FIG.17B) that provides services including distributed processing anddistributed storage.

The system 1700 may also provide a graphical user interface (GUI) thatallows users to specify how biometric data from different biologicalsystems is to be mapped to sounds, instruments, or soundscapes. The usermay provide inputs indicating which types of biometric data should bemapped to which musical elements, or which musical genres should be usedwhen mapping biometric data to musical elements. The GUI allows users tovisualize the accessed biometric data 1516, the encoded biometric data1506 and/or the generated state song 1511. Each state song may have acorresponding label that indicates its intended state 1512 (i.e. thestate into which the desires to transition). The GUI may also include agraph that is an isomorphic representation of the biometric data. Theisometric model is a virtual or digital twin of the user, and may beused (and reused) to generate state songs for the user. The user mayalso use the GUI to indicate which endogenous music (i.e. biometric,personal music) is to be combined with which exogenous music to generatea given state song. The GUI thus allows for a variety of inputs that cancontrol the generation of the state song.

Thus, using the systems described herein, biometric data may begathered, encoded and mapped to musical elements to create a state song.The state song can induce the user to a given mental or physical state.The state song may be based on an endocrinological model which spursinsulin generation in pancreas or spurs estrogen production to increasefertility, it may be based on a somatic model which reduces heart rate,blood pressure or stress, it may be based on a brain model whichincreases cognitive ability or clears the mind of unnecessary clutter.As can be seen, the state song is unique to each user, and can be usedto produce a variety of verifiable, measurable results in changing auser's state.

Many of the elements described in the disclosed embodiments may beimplemented as modules. A module is defined here as an isolatableelement that performs a defined function and has a defined interface toother elements. The modules described in this disclosure may beimplemented in hardware, a combination of hardware and software,firmware, or a combination, all of which can be behaviorally equivalent.Modules may be implemented using computer hardware in combination withsoftware routine(s) written in a computer language. It may be possibleto implement modules using physical hardware that incorporates discreteor programmable analog and/or digital hardware. Examples of programmablehardware include computers, microcontrollers, microprocessors,application-specific integrated circuits, field programmable gatearrays, and complex programmable logic devices.

As noted above, the application may be software embodied on a computerreadable medium which when executed by a processor component of acomputer device performs a sequence of steps. The application may be amobile application or application software configured to run onsmartphones, tablets computers, smart watches, and/or other mobiledevices. Moreover, embodiments of the present disclosure may comprise orutilize a special-purpose or general-purpose computer system thatincludes computer hardware, such as, for example, one or more processorsand system memory, as discussed in greater detail below. Embodimentswithin the scope of the present disclosure also include physical andother computer-readable media for carrying or storingcomputer-executable instructions and/or data structures. Suchcomputer-readable media can be any available media that can be accessedby a general-purpose or special-purpose computer system.Computer-readable media that store computer-executable instructionsand/or data structures are computer storage media. Computer-readablemedia that carry computer-executable instructions and/or data structuresare transmission media. Thus, by way of example, and not limitation,embodiments of the disclosure can comprise at least two distinctlydifferent kinds of computer-readable media: computer storage media andtransmission media.

Computer storage media are physical storage media that storecomputer-executable instructions and/or data structures. Physicalstorage media include computer hardware, such as RAM, ROM, EEPROM, solidstate drives (“SSDs”), flash memory, phase-change memory (“PCM”),optical disk storage, magnetic disk storage or other magnetic storagedevices, or any other hardware storage device(s) which can be used tostore program code in the form of computer-executable instructions ordata structures, which can be accessed and executed by a general-purposeor special-purpose computer system to implement the disclosedfunctionality of the disclosure.

Transmission media can include a network and/or data links which can beused to carry program code in the form of computer-executableinstructions or data structures, and which can be accessed by ageneral-purpose or special-purpose computer system. A “network” isdefined as one or more data links that enable the transport ofelectronic data between computer systems and/or modules and/or otherelectronic devices. When information is transferred or provided over anetwork or another communications connection (either hardwired,wireless, or a combination of hardwired or wireless) to a computersystem, the computer system may view the connection as transmissionmedia. Combinations of the above should also be included within thescope of computer-readable media.

Further, upon reaching various computer system components, program codein the form of computer-executable instructions or data structures canbe transferred automatically from transmission media to computer storagemedia (or vice versa). For example, computer-executable instructions ordata structures received over a network or data link can be buffered inRAM within a network interface module (e.g., a “NIC”), and theneventually transferred to computer system RANI and/or to less volatilecomputer storage media at a computer system. Thus, it should beunderstood that computer storage media can be included in computersystem components that also (or even primarily) utilize transmissionmedia.

Computer-executable instructions comprise, for example, instructions anddata which, when executed at one or more processors, cause ageneral-purpose computer system, special-purpose computer system, orspecial-purpose processing device to perform a certain function or groupof functions. Computer-executable instructions may be, for example,binaries, intermediate format instructions such as assembly language, oreven source code.

Those skilled in the art will appreciate that the disclosure may bepracticed in network computing environments with many types of computersystem configurations, including, personal computers, desktop computers,laptop computers, message processors, hand-held devices, multi-processorsystems, microprocessor-based or programmable consumer electronics,network PCs, minicomputers, mainframe computers, mobile telephones,PDAs, tablets, pagers, routers, switches, and the like. The disclosuremay also be practiced in distributed system environments where local andremote computer systems, which are linked (either by hardwired datalinks, wireless data links, or by a combination of hardwired andwireless data links) through a network, both perform tasks. As such, ina distributed system environment, a computer system may include aplurality of constituent computer systems. In a distributed systemenvironment, program modules may be located in both local and remotememory storage devices.

Those skilled in the art will also appreciate that the disclosure may bepracticed in a cloud computing environment. Cloud computing environmentsmay be distributed, although this is not required. When distributed,cloud computing environments may be distributed internationally withinan organization and/or have components possessed across multipleorganizations. In this description and the following claims, “cloudcomputing” is defined as a model for enabling on-demand network accessto a shared pool of configurable computing resources (e.g., networks,servers, storage, applications, and services). The definition of “cloudcomputing” is not limited to any of the other numerous advantages thatcan be obtained from such a model when properly deployed.

A cloud computing model can be composed of various characteristics, suchas on-demand self-service, broad network access, resource pooling, rapidelasticity, measured service, and so forth. A cloud computing model mayalso come in the form of various service models such as, for example,Software as a Service (“SaaS”), Platform as a Service (“PaaS”), andInfrastructure as a Service (“IaaS”). The cloud computing model may alsobe deployed using different deployment models such as private cloud,community cloud, public cloud, hybrid cloud, and so forth.

Some embodiments, such as a cloud computing environment, may comprise asystem that includes one or more hosts that are each capable of runningone or more virtual machines. During operation, virtual machines emulatean operational computing system, supporting an operating system andperhaps one or more other applications as well. In some embodiments,each host includes a hypervisor that emulates virtual resources for thevirtual machines using physical resources that are abstracted from viewof the virtual machines. The hypervisor also provides proper isolationbetween the virtual machines. Thus, from the perspective of any givenvirtual machine, the hypervisor provides the illusion that the virtualmachine is interfacing with a physical resource, even though the virtualmachine only interfaces with the appearance (e.g., a virtual resource)of a physical resource. Examples of physical resources includingprocessing capacity, memory, disk space, network bandwidth, mediadrives, and so forth.

The present disclosure may be embodied in other specific forms withoutdeparting from its spirit or essential characteristics. The describedembodiments are to be considered in all respects only as illustrativeand not restrictive. The scope of the disclosure is, therefore,indicated by the appended claims rather than by the foregoingdescription. All changes which come within the meaning and range ofequivalency of the claims are to be embraced within their scope.

While various aspects and embodiments have been disclosed herein, otheraspects and embodiments are contemplated. The various aspects andembodiments disclosed herein are for purposes of illustration and arenot intended to be limiting. Additionally, the words “including,”“having,” and variants thereof (e.g., “includes” and “has”) as usedherein, including the claims, shall be open ended and have the samemeaning as the word “comprising” and variants thereof (e.g., “comprise”and “comprises”).

While various aspects and embodiments have been disclosed herein, otheraspects and embodiments are contemplated. The various aspects andembodiments disclosed herein are for purposes of illustration and arenot intended to be limiting. Additionally, the words “including,”“having,” and variants thereof (e.g., “includes” and “has”) as usedherein, including the claims, shall be open ended and have the samemeaning as the word “comprising” and variants thereof (e.g., “comprise”and “comprises”).

1. A method, implemented at a computer system that includes at least oneprocessor, for generating and playing back a personalized state songrepresenting a target state of a user, the method comprising: accessingone or more portions of biometric data corresponding to one or morebodily systems of a user; encoding the one or more portions of biometricdata into data structures that are configured for modeling using analgorithm that is specific to the user; modeling the encoded biometricdata to identify one or more specified patterns in the encoded biometricdata, the identified patterns being used to generate a personalizedalgorithm that corresponds with the identified patterns unique to theuser; synthesizing a state song using the personalized algorithm, thestate song representing a specific biometric state for at least one ofthe bodily systems; and playing back the state song to the user toinduce the user to the specific biometric state.
 2. The method of claim1, wherein modeling the encoded biometric data further comprises:identifying one or more relationships between different types ofbiometric data; wherein generating the personalized algorithm implementsas input the identified relationships between the different types ofbiometric data.
 3. The method of claim 2, wherein the different types ofbiometric data are received from specialized biomedical devices.
 4. Themethod of claim 1, wherein the encoding is performed at a transcodingengine that includes an encoding layer configured to convert thebiometric data into musical elements by mapping the biometric dataaccording to at least one of sampling rate, filters, reverb orcompression.
 5. The method of claim 4, wherein a default mapping isestablished for each type of biometric data.
 6. The method of claim 4,wherein mapping includes mapping each specific type of biometric data toa specific instrument or soundscape.
 7. The method of claim 6, whereinan artificial intelligence engine is implemented to learn musicalpatterns that make music acoustically desirable, and wherein the learnedmusical patterns are used when mapping the different types of biometricdata to the instrument or soundscape.
 8. The method of claim 1, whereinthe state song is played back to the user in reaction to the userreaching a specified, undesirable biometric state.
 9. The method ofclaim 1, wherein the state song is played back to the user proactivelyto prevent the user from reaching a specified, undesirable biometricstate.
 10. The method of claim 1, further comprising rendering a graphthat is an isomorphic representation of the one or more portions ofbiometric data.
 11. A computer system, comprising: one or moreprocessors; system memory; a data accessing module configured to accessone or more portions of biometric data representing operations of one ormore bodily systems of a user; an encoding layer configured to encodethe one or more portions of biometric data, the encoding preparing thedata for modeling at a modeling layer; the modeling layer configured toidentify one or more specified patterns in the encoded biometric data,wherein the identified patterns are used to generate a personalizedalgorithm that corresponds with the identified patterns unique to theuser; a synthesis layer that uses the personalized algorithm tosynthesize a state song representing a specific biometric state for atleast one of the bodily systems; and a playback module that plays thegenerated state song back to the user to induce the user to the specificbiometric state.
 12. The computer system of claim 11, whereinidentifying specified patterns in the encoded biometric data furthercomprises identifying noise in the biometric data.
 13. The computersystem of claim 11, further comprising a filtering layer that isconfigured to filter at least a portion of the identified noise in thebiometric data.
 14. The computer system of claim 11, further comprisingan artificial intelligence layer configured to examine one or moreexisting songs and provide one or more musical elements as input to thesynthesis layer indicating how the state song is to be generated. 15.The computer system of claim 11, further comprising a sharing layerconfigured to provide the generated state song to one or more otherusers.
 16. The computer system of claim 11, further comprising agraphical user interface (GUI) that allows users to specify howbiometric data from different biological systems is to be mapped tosounds, instruments, or soundscapes.
 17. A method, implemented at acomputer system that includes at least one processor, for controllingstate dependent behaviors of a user, the method comprising: obtainingbiometric data from one or more data capture devices placed in contactwith a user; associating at least some of the obtained biometric datawith a specified instrument or sound; compiling at least some of theinstruments or sounds into a composition or song arranged to represent atargeted state; and providing the composition or song back to the userto induce the user to the targeted state.
 18. The method of claim 17,wherein endogenous music is combined with exogenous music within thecomposition or song.
 19. The method of claim 17, further comprisingproviding one or more genre filters for the obtained biometric data. 20.The method of claim 17, wherein the biometric data is moved to a serviceoriented architecture (SOA) that provides one or more services includingdistributed processing and distributed storage.